Selective constraint dominates the evolution of genes expressed in a novel reproductive gland.

One striking pattern in molecular evolution is that genes encoding proteins involved in reproduction tend to evolve rapidly. Seminal fluid proteins frequently exhibit this pattern and directly affect multiple reproductive processes including enhancing sperm performance and mediating postmating sexual selection. Here, we investigate molecular evolutionary patterns of genes expressed in the foam gland of Japanese quail (Coturnix japonica), a novel reproductive phenotype. Foam provides an interesting contrast to seminal fluid because it plays a similar functional role, yet is produced, stored, and transferred to females independent of semen. We combined RNA-Seq and comparative genomics to examine evolutionary rates of genes with enriched expression in the foam gland of Japanese quail and those that exhibit enriched expression in two other tissues (testis and liver) and with broadly expressed genes. Overall, we found pronounced heterogeneity in evolutionary rates. Foam gland genes evolved under strong evolutionary constraint, whereas testis genes evolved rapidly and sometimes adaptively. These striking differences were robust to variation in gene expression. Genes with enriched expression in the foam gland did not show major shifts in selective pressure after the quail and chicken lineages split; in contrast, testis-expressed genes experienced a burst of accelerated evolution specifically along the Coturnix lineage. Our work demonstrates that, as a class, genes expressed in the novel foam gland experience different selection regimes than genes expressed in many other tissues producing seminal fluid proteins. Our results also highlight the importance of selective constraint in shaping the evolution of male reproductive genes.